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ABSTRACT 

This  paper  examines  methods  of  rescunpling  for  bootstrap  from  a  survey  sampling  point  of 
view.  Given  an  observed  sample  of  size  n,  resampling  for  bootstrap  involves  n  repeated  trials 
of  simple  random  sampling  with  replacement.  From  the  point  of  view  of  information  content, 
it  is  well  known  that  simple  random  sampling  with  replacement  does  not  result  in  samples 
that  are  equally  informative  (see  Pathak  ( 1964)).  This  is  due  to  different  numbers  of  distinct 
observations  occuring  in  different  bootstrap  samples.  We  propose  an  alternative  scheme  of 
sampling  sequentially  (with  I'eplacement  each  time)  tintil  k  distinct  original  observations 
appear.  In  such  a  scheme,  the  bootstrap  sample  size  i:)ecomes  random  as  it  varies  from 
sample  to  sample,  but  each  sample'  has  exactly  the  same  number  of  distinct  ob.sou’vations. 
We  show  that  the  choice  of  A:  =  (1  —  c"'  )n  ~  .632??.  has  some  advantage,  stemming  from 
the  observation  made  by  Efron  (1983)  that  the  usual  bootstrap  samples  are  supported  on 
approximately  .632n  of  the  original  data  points.  Using  recent  results  on  empirical  processes, 
we  show  that  main  empirical  characteristics  of  the  sequential  resampling  bootstrap  are 
asymptotically  within  the  distance  of  order  ~  of  the  corresponding  characteristics  of 

the  usual  bootstrap. 


Keywords  and  phrases:  bootstrap,  sequential  resampling,  information  content,  sampling  viewpoint, 
asymptotic  correctness,  empirical  measures,  empirical  processes,  weak  convergence,  symmetrization,  Pois- 
sonization,  p- variation,  quantiles 
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1.  INTRODUCTION 

Let  <S  =  (Xi,-Y2,...  .A’„)  be  a  random  sample  from  a  distribution  F,  and  9{F)  be  a 
given  parameter  of  interest.  Let  F„  denote  the  empirical  distribution  function  based  on  5, 
and  suppose  that  0(F,i)  is  to  be  used  as  cin  estimator  of  d{F).  Then,  for  a  large  class  of 
functions  0,  Efron’s  bootstrap  resampling  method  (1979.  1982)  provides  a  robust  method 
of  evaluating  the  performance  characteristics  of  9{Fn),  solely  on  the  basis  of  information 
derived  (through  randomization)  from  the  observed  sample  S.  As  an  illustrative  example, 
consider  the  simple  case  of 

0{F)  =  J  xdF  ^  ^i{F),  say.  (1.1) 


Let 

aHF)  =  Ji.c-fdF)fdF  (1.2) 

li„  =  -yx,  (1.3) 

n 

and 


-  dnf  (1.4) 

n. 

Then  //.„  =  //.(F„)  and  rrf,  =  rr-[F„).  L<'t 

n„  =  v^(//.„  - /0/cr„  (1.5) 

=  {,dF„)-i,{F))ly/<jHF,)  (1.6) 

The  central  limit  theorem  entails  that  the  sampling  distribution  of  II,,  can  be  approximated 
by  the  standard  normal  distribution.  On  the  other  hand,  the  bootstrap  method  furnishes  an 
alternative  approach  to  estimating  the  sampling  distribution  of  fin  (by  repeated  resampling 
of  the  observed  sample  S)  as  follows: 

Given  the  observed  sample  S.  select  a  simple  random  sample  with  rcplcicement  (SRSWR.) 
of  size  11  from  the  observed  sairiph'  S.  Let  <S„  =  (A'l , . . .  ,  A',,)  denote  a  sample'  drawn  from 
S  in  this  fashion.  Let  F„  Ix'  the  empirical  distril)ution  based  on  S„.  Let 

n«  =  \/n(fl(Fn)  -  fl(Fn))l\jcr'^{Fn) 

=  say.  (1.7) 

Then  for  large  ??.,  the  conditional  distribution  of  n,,  given  S  is  close  to  that  of  !!„.  Thus, 
one  can  replace  the  sampling  distribution  of  II,,  by  the  conditional  distribution  of  II„  given 
S.  In  practice,  the  conditional  distribution  of  II„  is  estimated  by  the  frequency  distribution 
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(ensemble)  of  n„  obtained  by  repeated  resamplings  of  S  by  SRSWR  a  large  number  of 
times;  the  observed  sampling  distribution  (jf  !!„  is  referred  to  as  the  bootstrap  distribution 
of  n„.  Thus  for  example,  for  large  the  quantiles  from  the  bootstrap  distribution  of  n,t 
can  be  used  to  set  up  a  confidence  interval  for  /.i  based  on  the  pivot  n„. 

It  is  well  known  that  owing  to  the  with  replacement  nature  of  SRSWR.  not  all  of  the 
observations  in  bootstrap  samples  <Sn,  say,  will  be  based  on  distinct  observations  from  S. 
In  fact,  the  information  content  of  <S,i,  the  set  of  distinct  observations  in  <Sn,  is  a  random 
variable.  Let  i/„  denote  the  number  of  distinct  observations  in  S„.  Then  it  is  easily  seen 
that 


El;/,,)  =  n[l -(!--)«]  (1.8) 

~  7?[1  —  C“'] 

~n(.G32).  (1.9) 

Var(  )  =  \d>.r(  ii  -  u„  ) 

-  7).(1  -  -  )"  +  n(n  -  1)(1  -  -  )"  -  n-(l  -  -)'^"  (1.10) 

v  n  71 

~  +  ii(ii  —  l)c~^  —  n^(~^ 

~  7;,f:"‘(l  -  f-') 

~  77.(.3C8)(.C32)  ~  77, (.233)  (1.11) 

so  that 

7T(;/„)  ~  .482v^.  (1.12) 


In  fact,  approjiches  a.  binomial  distril)ution  h{ii.p)  with  p  ~  .032.  This  shows  that  in  !i 
bootstrap  sample,  approximatc'ly  sixty  p('rcent  of  tlu*  information  from  the'  oI)s('rved  saniph' 
S  is  utilized  while  the  n'st  of  the'  forty  pc'rcent  of  the  data  in  it  is  simply  randomization. 
Besides,  the  2(T-limits  for  are  rvpproximately  (.032)77  ±2(. 482) \/?7  ~  (.032)77  ±  y/ii.  There¬ 
fore  in  approximately  95%  of  rc'samplings,  the  ('ssential  supremum  of  can  be  as  large 
as  (.032)77.  -f  y/n.,  and  the  essential  iiifimum  as  small  as  (.032)77  —  y/n.  Thus  for  n=l()0. 
this  means  that  one  can  expect  u,,  to  roughly  vary  in  the  range  (53,  73).  Viewed  from  a 
purely  information  content  of  the  bootstrap  samples,  this  variability  is  neither  necessary  nor 
desirable.  As  an  alternative  to  a  fixed  size  l)ootstrap  samples,  we  examine  in  what  follows 
the  following  sequential  api)roa,ch. 

2.  A  SEQUENTIAL  RESAMPLING  APPROACH 

To  select  a  bootstrcip  sample,  draw  observations  from  <5  .sequentially  by  SRSWR  until 
there  are  (r7i  +  1)  w  77(1  —  )  +  1  distinct  observations  in  the  ol^served  bootstrap  sample, 

the  last  observation  in  the  bootstrap  sample  is  discarded  to  ensure  simplicity  in  technical 
details.  Thus  an  observed  bootstrap  sample  has  the  form: 

<S/v  =  ( A'l ,  A'2, . . .  ,  A'/\/ ) 


(2.1) 
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in  which  Xi , . . .  ,  Xp^  has  777  ^  77  ( 1  —  €  ^ )  distinct  observations  from  S. 

The  number  of  distinct  ol)servations  in  5/v  is  precisely  [7r(l  —  it  is  no  longer  a 

random  variable.  It  is  worth  noting  that  the  size  N  of  the  bootstrap  sample  is  a  random 
variable  with  E{N)  ~  n.  The  pivot  l)ased  on  Sr\i  is 

n/v  =  \/iV(/dT’,v)-/di^«))  (2.2) 

in  which  F/v  denotes  the  empirical  distribution  based  on  the  bootstrap  sample  Ss-  Again 
for  simplicity  in  the  exposition,  we  assume  that  =  1.  Now  for  a  comparative  study 

of  the  pivot  II/v,  based  on  the  s('quential  resampling  approach,  versus  Il„,  based  on  a 
bootstrap  sample  of  fixed  size  .  it  is  necessary  to  estimate  the  order  of  magnitude  of  the 
random  varicible  N .  It  is  ('asily  s('en  that  N  admits  the  following  decomposition  in  t('rms  of 
independent  (geometric)  random  variables: 


-V  =  /, 

in  which  iii  =  [//(I  —  e“' )]  using  tlu'  notation  [•]  to  indicate  the  largest  int(’g<'r:  I\ 
for  ('ach  k.  2  <  k  <  ni. 

r.,r  k-1  k-1 

P(4  =,/)  =  (! - )(  — 


p-i 


71 


for  j  =  1.2,....  Therefon' 


E{N)  =  ^ 


71  (71  —  1 ) 


+  ...  + 


(n-7n  +  1)'* 


(2.3) 

1.  arid 

(2.4) 

(2.5) 


It  can  be  seen  that 


E{N)  =  77  log 


77 


[77  —  777, 

=  77  +  0(1) 


+  0(1) 


sinc<'  777  =  [n(l  —  r  *  )].  Siuiihirly.  it  is  <'<isily  seen  that 

77(A:-1) 


Var(iV)  =  ^ 


(77-  A- +  1)'^ 


*,=  1 

=  n{e  -  1)  +  0(1). 


(2.C) 


(2.7) 


(2.8) 


Thus 


E(N  -  7/.f 


(r:-l) 

77. 


+o(-4) 

77^ 


(2.9) 


showing  that  (iV/u)  — »  1  in  i)rol)!ibility.  Further  analysis  similar  to  that  of  Mitra  and  Pathak 
(Mitra,  S.K.  and  Pathak,  P.K.  [1984])  ciin  be  used  to  show  that 


EjtlN-flnf  ^  /Var(iV) 
Var(n„)  ~^^\l  77.^ 


(2.10) 

(2.11) 
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This  implies  that  n„  and  H/v  are  asymptotically  equivalent.  The  preceding  computations 
show  that  the  sequential  resampling  plan  introduced  here,  in  addition  to  keeping  the  in¬ 
formation  content  of  bootstrap  samples  constant,  also  preserves  its  asymptotic  correctness. 
With  this  heuristic  introduction  to  the  sequential  resampling  scheme,  we  turn  now  to  the 
question  of  mathematical  justification  of  its  validity  with  the  main  goal  of  showing  that 
the  sequential  bootstrap  and  the  usual  one  are  at  a  distance  which  is  enough  to 

justify  its  asymptotic  first  order  correctness. 

Let  r2  be  the  set  of  probability  distributions  with  finite  second  moment.  We  say  that 
distributions  {F,,  }  converge  in  M-sense  to  F  if  and  only  if  (a)  Fn  converges  weakly  to  F 
and  (b)  f  x^dF,,  converges  to  f  x^dF.  It  is  easily  seen  that  this  notion  of  convergence  is 
induced  l)y  the  following  A/-nietric‘*  on  F^.  For  each  F,  G  in  F^,  define  the  squared  distance 
(f{F,G)  between  F  and  G  as  follows: 


(P{F,G) 


inf  E(X-Yf 

X^F.Y-^Cl 


(2.12) 


in  which  the  infimuin  is  taken  ov('r  all  pairs  of  random  variables  ( A’,  1)  with  given  marginals 
F  and  G  r(\sp(H-tiv('ly  (Mallows  (1972)). 

Now  let  Ti , . . .  ,  y,„  b('  indc'pc'iident  random  variabh's  with  a  common  distribution  G.  Let 
G("')  1)0  the  distribution  of  the  standardized  varial)l(": 

1 

•Sm  =  v/^{- yd'  -F(y}))}.  (2.13) 

7n  ^ ^ 

7=1 


Then  the  following  result  holds: 

Lemma  2.1  (Mallows). 

r/(G'<"'*.iF‘'”*)  <  d{G,H).  (2.14) 

The  proof  of  the  validity  of  Efron's  ])ootstrap  leased  on  rescimpling  schenies  of  a  preassigned 
size  ni  can  now  l)e  sc^en  to  Ix'  a  consequence  of  the  following  triangle  inequality: 

+  d{F^^\^)  (2.15) 

<d(F„,F)-hd(F<^>,$).  (2.16) 

(Note  that  F//"*  here  stands  for  the  conditional  distribution  of  the  jjivot  fl,,,  (eq.(1.7)),  and 
F*'”'  stands  for  the  distribution  of  n„,  based  on  a  sample  of  size  m  (eq.(1.5)).) 

In  this  inequality  the  couv('rgenc('  of  d(F*"'*,  <&)  to  zero  essentially  follows  from  the  central 
limit  theorem;  th('  convergence  of  d{Fu,F)  to  zero  is  essentially  a  conseqiunice  of  the  fact 
that  the  sample  mean  converges  in  mean  to  the  population  mean  (i.e.  lim  E(  J  xdFn  — 
J  xdF)^  =0).  Despite  the  apparent  simplicity  of  this  proof,  an  added  complication  that 
arises  in  sequential  resampling  scheme  is  that  the  bootstrap  sample  size  is  now  a  random 
variables;  it  is  denoted  by  iV.  Lemma  2.1  is  no  longer  directly  applicable.  Nevertheless  the 
following  extensions  of  Lemma  2.1  hold. 


^ known  also  in  the  literature  as  a  Wjisserstein  metric;  see,  e.g.  Rachev  (1984) 


< 


Lemma  2.2. 


,2 

E{(^{fiN  -  fin)  -  V^ifiN  -  /%))''|<5}  <  A'l-  (2.17) 

n 

where  is  the  sample  variance  based  on  the  initial  sample  and  K\  is  a  universal  constant. 

Lemma  2.3 


E[n[fiN  -  fin)^\S]  <  (2.18) 

\/tl 

The  pro(jfs  of  these  lemmas  a.re  l)ase(l  on  teelmiciues  similar  to  those  of  Pathak  (1964)  and 
Mitra  and  Patlmk  (1984)  (('f  Lemma  3.1  in  Mitra  and  Pathak). 

Lemmas  2.2  and  2.3  imply  that  the  Mallows  distance  l)etween  \/N fiisr  <uid  converges 

to  zero.  This  then  inipli('s  the  asymptotic  corr(H‘tne.ss  of  v/iV/l/v.  iiiid  thus  concludes  tlu' 
heuristic  justification  for  the'  corrc'ctiu'ss  of  the  sample  sum  for  tlu'  se'cpK'iitial  l)oot.strap. 

In  the  next  section  we  c'xamiiK'  the  mathematical  justification  for  the  prex-eding  heuristicr 
discussion  in  some  detail. 

3.  MATHEMATICAL  JUSTIFICATION:  BOOTSTRAPPING  EMPIRICAL 
MEASURES  WITH  A  RANDOM  SAMPLE  SIZE 

Let  (A'.  A,  P)  l)e  a  probability  spcice.  and  let  (II,  E,  7)  be  the  product  of  a  countable 
number  of  copies  of  (A’,  A.  P)  and  the  unit  interval  with  the  Lebesgue  measure.  In  what 
follows  E  denotes  the  expc'ctation  with  respect  to  7.  Consider  a  seciuo'iux'  {A’„  :  n  >  1}  of 
random  elements  defined  on  (fI,E.7)  as  follows: 

A„(^’)  =  ''‘«  (3.1) 

in  which  u>  —  (x) ,  .v  ? , . . .  .  a )  €  fl.  Then  { A'„  :  11  >  1 }  is  a  sequence  of  indepc’iident  random 
elements  with  a  common  distribution  P.  Let  P„  denote  the  empirical  measure  based  on  the 
sample  ( A’l , ....  X„  )  from  P.  i.e., 


P„:=n-'  Y,  '^-V.  (3.2) 

1  <  I  <  It 

with  Sx  being  the  unit  point  mass  at  x  G  A'. 

Now  lc;t  {A*„,j  :  ;  >  1}  Ix'  a.  sc'cpu'uce  of  independcnit  random  elements  with  common 
distribution  P„;  assume  that  this  sequence'  is  defined  on  (iI.E,7)  with  E  denoting  the 
expectation  with  respect  to  7.  We  refer  to  this  sequence  as  a  sequence  of  bootstrapped 
observations,  or  just  a  Ijootstrapped  sequence.  Given  a  number  N  >  1,  let  P,i,/v  denote  the 
empirical  measure  based  on  the  bootstrap  sample  (A'„7, . . .  .Xn,N)  oi  size  iV);  i.e., 

Pn.N:=N-'  Y  h 

l<i<N 


(3.3) 
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We  refer  to  it  as  a  bootstrapped  empirical  measure  of  size  N. 

In  what  follows  we  assume  that  N  =  iV„  is  a  random  variable  defined  on  the  space 
(fi,E,7)  X  (0,  E,7)  and  takes  only  positive  integer  values.  Thus  we  allow  the  size  of  the 
bootstrapped  sample  to  l)e  a  random  variable.  The  main  object  of  this  section  is  to  show 
that  if  (iV/n)  converges  to  one  in  prol)ability,  then  the  bootstrapped  empirical  measure  P„jv 
is  at  a  distance  of  from  the  bootstrapped  empirical  measure  P„  ;=  Ai,;n  so  that 

all  of  the  \/n  -  asymptotic  results  for  the  classical  bootstrap  carry  over  to  the  sequential 
bootstrap  with  a  random  sample  size.  To  do  so,  we  need  to  introduce  a  few  preliminary 
results  on  limit  theorems  for  general  ('inpirical  processes  (see  Dudley  (1984).  Gine  and  Zinn 
(1986  and  1994)  and  Pollard  (1984)). 

A  gent^ral  empirical  procc'ss  is  (U'fiiK'd  a.s  a  random  signed  measure; 

Z„  ;=  v^(P„  -  P),  u  >  1,  (3.4) 

on  the  measurable  space'  (A", >1).  Give'ii  a  measurable'  functie)n  /  and  a  sigueel  measure  A 
em  (A", >1),  let  A(/)  ele'iie)te'  the  iiite'gral  / /e/A.  pren-ieleel  j  |/|e/|A|  <  oo.  It  is  e-e)iiveuiie'nt 
te)  e-e)nsieler  the  e'lnpirical  pre)ce'ss  Z„  as  a  ranele)m  fune'tie)n  f  Zn(f)  ele'fine'el  e)n  a  class 
T  C  L-iiX:  (IP).  The'  finite  dime'nsioiial  elistributiems  e)f  the  se'epie'iie'e;  of  raiielom  functiems  Z„ 
converge  tej  the  finite  dime.'iisieHial  elistributiems  e)f  a  Gaussian  randejin  functiem  /  ^  Gp[f) 
with  mean  zero  and  covariance'  functie)n 

D{Gp(f),G,>((j))  =  Pifej)  -  P[f)PGj).  .f.cj  6  L^iXulP).  (3.5) 

Such  a  Gaussian  random  function  is  eTten  calkd  a  P-Brownian  bridge.  We  e'lidejw  L-iiX:  dP) 
with  a  metric  f)p  elefinexl  by 


:  =  D{Gp{f)  -  Gp{(,),Gp(f)  -  Gp(g)) 

=  Pi.f-!l)~-{P{.f -!/)){  (3.6) 

Coiiverpjciice  of  the  srx[ncncc  {Z„  :  n  >1}  to  the  limit  Gp  (now  in  functional  sense)  has 
been  studied  (see,  for  example',  Pollard  (1984))  in  the  space  of  all  imiformly  hoi.mded 

functions  on  the  class  T  with  tlu'  norm 

||F|l^  :=  sup{|F(/)|  :  /  G  P},  F  G  e°(P).  (3.7) 

Zn  belongs  to  this  space  a.s.  under  the  condition 

/  \\^r\\j^P{(G-)  =  /  .sup  |/(.r)|P(r/./:)  <  +00.  (3.8) 

./.V  ./.Y  fei  ' 

Let  Chuid-)  be  the  subspace  e)f  all  p/>-uniformly  continuous  and  uniformly  be)unded  functions 
on  p". 

In  the  sequel,  we  use  weak  convergence  of  random  functions  in  the  space  i°°{P)  in  the 
Hoffman- J(;irgensen  sense.  Let  1)^  a  sequence  of  functions  from  Q  x  p  into  such  that 
for  all  n  >  1 


7*({w  :  Hulk  =  =  0 


(3.9) 
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in  which  7*  denotes  the  outer  measure  induced  by  7  (note  the  need  for  7*  since  C,,i  is  not 
necessarily  a  random  element  (measurable)).  Then  the  sequence  is  said  to  converge 
weakly  in  to  a  limit  (^  :  Q  x  J-  iff 

(a)  7*({C^C',,„(:F)})=0: 

(b)  for  any  bounded  and  ||  •  HjF-continuous  functional  $  :  i?‘,  we  have 

£'*$((„)  — >•  as  n  — >  00 

in  which  E*  is  the  outer  expectation  induced  by  the  expectation  operator  E.  The  main 
results  of  the  classical  theory  of  weak  convergence  of  continuous  processes  hold  for  this 
extension  as  well;  of  particular  interest  is  the  following  result:  Let  BLi((''^{E))  be  the  set 
of  all  functionals  $  :  7?'  such  that 

||$||oo  :=  Hup  |<i'(y)|  <  1 


and  for  all  yi,  (/2  € 

|<I>(yi )  -  ‘I*(.y2)|  <  ||yi  -  inWr- 

Given  two  functions  1 1  Ci  :  f ^  7?  ’ .  <lefine  the  following  distance  between  G  and  G  • 

^MCi.G)-  -sup  |E*$(Ci)-£:*$(C2)|.  (3.10) 

Then  a  sequence  {Gi  :  »  >  1}  converges  weakly  in  the  space  i°°{E)  to  the  limit  if  and 
only  if 

7*({C^CWJ^)})  =  0 

and 

lim  cGIC/mC)  =  0. 

n  — -00 

A  class  T  C  L^iX:  (IP)  is  calU'd  P-pregaiissian  iff  there  is  a  version  of  P-Brownian  bridge 
Gf>  such  that 

7*({G'p^C’ft„(^)})  =  0. 

This  class  is  called  P-Donsker  iff  the  sequence  of  empirical  processes  Z„  converges  weakly 
in  the  space  ll'^(E)  to  the  P-Brownian  bridge  Gp.  In  this  case  we  say  that  E  G  CLT{P). 
The  following  result  is  well  known  (Gine  and  Zinn  (1986)).  If  P  €  CLT(P).  then 

l<!<» 


It  is  also  clear  that  E  G  CLT{P)  iff  E  is  P-pregaussian  and 

^  G p)  — ^  0  as  77-  — ^  00. 

In  what  follows,  we  use  several  well  known  inequalities  for  sums  of  independent  random 
elements  in  Banach  spaces.  We  assume  that  all  random  variables  and  elements  in  the 
following  three  lemmas  are  defined  on  a  certain  probability  space  {Qq,'£q,Q).  We  denote 
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the  corresponding  expectation  operator  Eq.  Let  B  be  a  Banach  space.  We  start  with  the 
following  symmetrization  lemma  (Gine  and  Zinn  (1986)). 

Lemma  3.1  (Symmetrization  lemma):  Let  {F„  :  n  >  1}  be  a  sequence  of  independent 
and  identically  distributed  random  elements  in  B  with  finite  first  moment.  Let  {t,,  :  n  >  1} 
be  a  Rademacher  sequence  (i.e..  an  i.i.d.  sequence  with  Qis,,  =  — 1)  =  Qi^n  =  1)  =  1/2) 
independent  of  { F„  :  ??  >  1 } .  Then 


n  n 

(.)  £q||  J](r,  -  £r,)||  <  2£,;|| 

t=[  1=1 

and 

n  n 

(»)  £«|| <  2£q|| 

t=l  i=l 


Lemma  3.2  (Contraction  principle):  Let  :  ti  >  1}  and  {fjn  :  n.  >  1}  Ix'  two  i.i.d. 
sequenc(\s  of  syiniiK'tric  randoni  varia])l(\s  sncli  tliat,  for  ('ach  ^  >  0, 


C?({bh,|>^})<Q({len|>^}). 


Then  for  each  n  >  1  and  for  all  xi,X2, ...  ,Xn  G  -B. 

n  n 

^q\\  ^ 

I  I 

For  a  proof,  see  Ledoux  and  Talagrand  (1991). 

Lemma  3.3  (Poissonization  lemma):  For  each  ?;  >  1.  let  .ci. . v„  G  D:  l('t  {Vn.j  ’■ 

1  ^  ^  be  m  i.i.d.  R-valuc'd  random  elements  with  the  distribution 


n 


i={ 


Let  {ti  :  i  >  1}  and  {7rj^[7n/2??]  :  /  >  1}  respectively  lx?  a  Rademacher  sequence  and  a 
sequence  of  independent  symmetrized  Poisson  random  variables  with  parameters  inf'lii, 
l)oth  independent  of  :  1  <  ./  <  7/?  }.  Then 


?)£q||  <£5||^.-,r„,,||  <2£g||^< 

i=l  j=l  i=l 


in 


in 


(3.12) 


(see  Proposition  2.2  in  Gine  and  Zinn  (1990)  and  Lemma  4.2  in  Praestgaard  and  Wellner 
(1993)).  Finally,  we  recall  a  few  well  known  properties  of  the  symmetrized  Poisson  randoni 
variables.  Let  7ri[A],  i  =  1, 2,  be  two  independent  Poisson  random  variables  with  mean  A  >  0. 
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Then  7r^[A]  tti  [A]  —  Tr^fA]  is  a  symmetrized  Poisson  random  variable  with  parameter  A  >  0. 
It  is  easily  seen  that 


=  ^■})  = 

We  will  use  the  following  elementary  results: 


-2x,y  ^  y  k  =  Q  i‘-> 


(3.13) 


Lemma  3.4  (i)  If  0  <  A|  <  A2  <  1/2,  then,  for  each  t  >  0, 

Q({k1A,]|  >  t})  <  Q({|7r^[A2]|  >  f.}).  (3.14) 

(ii)  For  each  A,  0  <  A  <  1/2.  and  for  any  A’  =  0, 1,  2, . . . 

=  ^-l)  <  (3.15) 

Suppose'  now  that  :  //  >  1 }  is  a.  uoiiincreasing  se'ciuence'  of  classes  of  functions  in 

L‘>{X\  (IP),  i.('.  LiiX:  (IP)  3  .F;,  3  Pn-^-s  h>i‘  <dl  n  >  1.  Let  (/’  1)0  a  (■oucav<‘  uoiKh'cn'asiuf^ 
function  from  [0.  +oo)  into  [0.  +oo),  sucli  tliat  for  some  r  >  0  and  for  all  .r  >  0,  y  >  0 


ii'ixy)  <  (■tiix)t{y) 

(3.16) 

and 

OO 

1 

7=1 

(3.17) 

Lemma  3.5.  Suppose*  that  for  all  n  >  1 

n 

^*11  ~  -  '(’(«). 

(3.18) 

I 


Then  for  all  in  <  n 


k 

E*  max  .Ell  V' (6y  —  E„)||:f„  <  const  i/'(m). 

k<ni 

1 


(3.19) 


Proof:  To  e.stimate  the  expectation  in  (3.19),  we  need  the  following  sequences  of  mutually 
independent  random  variables: 

(a)  a.  Rademachcr  sequc'nct'  {cj  :  j  >  1}; 

(b)  for  each  k,l  <  A:  <  in,  a  sequence  {7r|[A‘/2/i]  :  j  >  1}  of  independent  symmetrized 
Poisson  random  variables  with  parameters  {k/2n). 

We  assume  that  the  Rademacher  secjuence.  symmetrized  Poisson  random  variables,  and 
the  bootstrapped  sequence  :  j  >  1}  are  all  independent  of  one  another.  More  specif¬ 

ically,  we  define  the  Rademacher  secpience  on  a  separate  probability  space  {Qg,Ee,"/s)\  the 
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symmetrized  Poisson  sequences  on  another  probability  space  (r2,r»  Ttt)-  Thus 
follows,  our  underlying  probability  space  admits  the  form: 

(as, 7)  X  (as, 7)  X  (a,S,,7e)  x  (Q,,S^,7^). 

Then  by  the  symmetrization  lemma,  we  luive  a.s.  (7) 


max  l;||  ^(^A7  -  ■Pn)||;r,.  <  2  max  EE,\\  ^ SjiS^  -  P„)||;c,. 

-  1  -  I 

which.  l)y  the  Poissonization  lemma,  is 

<  2  max  E^W  ^  <[— ||;r„ . 

k<7n  Ln 

And  now  by  the  eontraetion  i)rincii)l('  and  Lemma  3.4(i),  the  (expression  (3.21)  is 

<  2£’„||  ^  TT- [— ]A.v,  ||:r„. 

^ ^  In 

i=i 

Since  random  varicibles  7rf  arc  symmetric,  we  have 


i-\ 


rn 

2ri' 


<  EE,E,\\  E  E  l<l^l|Z«=-.*A',ll^„. 

1=1  J>1 


where  Ztj  =  1  if  =  ±/,  and  Z,j  =  0  otherwise.  Then  (3.23)  is 

11 

=  ee,e,\\y,j  E  =.*-v.h^..' 

J>>  '=l  J>1  A„(j) 

where  An(j)  :=  {«  :  k|'''’( ^ )|  =  i,  1  <  i  <  n).  Thus 

(3.24)  <  Y,jE.E,E\\ 

J>'  A,.(» 


Now  the  symmetrization  h’lnina  and  condition  (3. IS)  imply  that  for  K  :=  eard(  A,, 


£.E||  E  E  - -P)ll^. 

i€A„(j)  i€A„(j) 

K 

<  2E\\  -  P)y„  <  2iiK)  =  20(card(A„(i))). 

i=l 


in  what 

(3.20) 

(3.21) 

(3.22) 

(3.23) 

(3.24) 

(3.25) 

ij)) 


(3.26) 
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Therefore,  by  (3.26)  and  Jensen’s  inequality,  (3.25)  is 


<  2'Y^jE^4>{cnvd(An(j)))  <  2 ^ card( A„(j )) 

>>i 

n 

=  2Y,jiHE^Y.^d)  =  2Y,3iinP{Z,,  =  1)) 

j>i  1=1  j>i 

<  2  / 1/'( const  7?.(  —  )-^  ^),  by  Lemma  3.4(ii), 

■  2n  I ! 

j>^  • 

-  -  J  l/’(  const  77l(  ^  )-^  “  *  -^^ ), 

j>\  “  J- 

<  const  7r{7n),  l)y  (3. 16)  and  (3.17). 


Remark:  There  could  Ix'  anotlu'r  apjiroach  to  the  proof,  liased  on  Lemma  1.2.4  in  Gine 
and  Ziim  (1986)  and  Lc'inma.  4.4  in  Praxistgaard  and  Wx'llner  (1993).  But  we  preferred 
to  pres(>nt  here  a.  proof,  liasx'd  mostly  on  the  propx'rties  of  symmetriml  Poisson  random 
variabk's. 

The  investigation  of  tin?  classical  l)ootstrap  for  the  general  empirical  measures  was  ini¬ 
tiated  by  P.  Gaenssler  (1987).  The  following  remarkable  theorem  is  due  to  Gine  and  Zinn 
(1990). 

Theorem  3.1.  The  following  two  conditions  are  equivalent: 

(a)  EeCLTiP): 

(b)  There  (exists  a  Gaussian  process  G{f),  f  6  E.  defined  on  the  probability  space 
(f2,S.7)  such  that  G  G  CbuiE)  7-a..s.  and 

d-,(v^(A,  -P„),G)^0, 


in  probability  (7),  as  7/  — >  00.  The  proex'ss  G  is  a  P-Brownian  liridge. 

Praestgaard  and  Wellner  ( 1993)  studied  more  general  versions  of  the  bootstrap,  including 
the  one  in  which  the  size  of  Imotstrap  sample  tv.  ^  77. 

W'e  adopt  the  following  terminology  in  the  secpiel.  Given  a  sequence  ?/„  of  random  vari¬ 
ables  defined  on  the  probaliility  splice  (Q.  S,  7)  x  (ft,  S,  7)  and  a  sequence  o„  of  positive  real 
numbers,  we  write 

~  (3.2/ ) 

iff,  for  ('ach  £  >  0, 

7  X  7({|'/»i|  >  i'«»})  ->  0, 


as  77  —7  00. 
We  write 


^In  —  Gp{cLf, ) 


iff 

lim  limsup7  x  7({|?/n|  >  ca„})  =  0. 

C  ^00 
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It  is  easy  to  check  that  =  Oj,{a„)  if  and  only  if  r]n  =  Op{bn)-,  for  any  sequence  bn  of 
positive  numbers  such  that  a„  =  o{b„ ). 

An  immediate  consequence  of  Fubini's  theorem  is  that  =  Op{an )  iff,  for  each  £  >  0, 

7({w:b/„(-,c2;)|>£a„})->0,  (3.28) 

as  n  — »•  oo  in  probability  (7). 

The  following  theorem  furnishes  the  main  results  of  this  section. 

Theorem  3.2.  Let  {«,(}  be  a  sequence  of  positive  real  numbers  such  that  a„  =  0(n). 
Suppo.se  that  JF  €  CLT{P)  and  let 

\Nn-n\  =  Op(a„).  (3.29) 

Tluni 

-  p„y  =  op{—).  (3.30) 

n. 


Proof:  In  view  of  (3.28).  it  is  enough  to  show  that,  for  ('ach  £  >  0, 


idia. 


1/2 

SCln' 


}-.() 


(3.31) 


as  n  ^  00,  in  7. 

Let  Pf^n^n  cloiioto  the  (unpirical  ineasiire  based  on  —  /;|  hootstmpped  ()l>ser  vat  ions 
l)etweeii  71  and  .  Then,  we  hav(' 


Pu  j\\i  —  d"  ( 1  Y 


(3.32) 


which  implies 


lia,,v.,  -  PuWr  =  11-1^1  llCv.A,  -  aii^ 

-  ’  n 

-  p„l\r  <  "  )i||p,^  _  p_||^ 

11  iV,| 

+  ||A,AiV,.  --Pall^].  (3.33) 

From  (3.29),  it  follows  that  as  n  — >  oo,n/N„  =  1  +  Op(l),  and  \Nn  —  n\/n  =  Op{an~ fn). 
Moreover  the  Gine-Zinn  Theorem  (Theorem  3.1)  implies  that  for  any  sequence  ^  oo,  we 
have 

\\Pn-Pn\\j^  =  0^(^)  (3.34) 

sjll 


in  probability  (7). 
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Thus,  under  the  condition  cin  =  0{n),  we  have 


1  /2 

liVn  -  n.|  n  ^  .  /“"MID  D  II  _  \ 

yr  -Pfl  )  -^n  —  ^p(  )■ 

n  iV„  n  ‘  n 

Therefore  it  suffices  to  show  that 

\^n  —  »|||Ai.AN„  —  Pn\\j^  =  0^(a\J'^) 

in  probability  (7). 

Now  let  rUn  =  [«ni'']-  Then 

7({|W„-n|||P„„A„-P,.ll;r>Eai7)) 

=  7({|A'»  -  »II|P|«.-..|  -  C,||^  >  ia'J-‘)) 

\N-„\ 

=  i((il  E  e.v„„  -c,)i|y  >£a;7)) 

1 

<  -  "I  >  '"»}) 

k 

+  7({  II  ^  ). 


(3.35) 


(3.36) 


By  condition  (3.29),  we  have 


7({l-^n  -  «l  >  '«n)  ^  0 


(3.37 


(3.38) 


in  probability  (7). 

Now  to  estimate  the  last  term  in  (3.37),  we  invoke  the  following  version  of  Ottaviani's 
inequality: 


K  {  II  ^( h  ^  -  P„  )||:p  >  za\l'^ } ) 

k<Wn 

7((iie';"'('',v.,  -a)ik>e<.y7) 

1/2 


< 


1  -  7(  { II  E 1  )ll^  ^  } ) 

Thus  it  is  enough  to  show  that 

k  _  1  /2 

max  7(  { II  5^(^  v„  ,  -  )ll^  ^  } )  -^  0, 

in  probability  (7).  Clearly 


(3.39) 


(3.40) 


'/2 


max7({||  ^(<^'.v,.  .  -  Pn)\\T  >  ^^^}) 


(3.41) 
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To  estimate  this  last  expectation,  we  apply  Lemma  3.5  to  the  classes  Tn  T  and  t/»(x)  := 
const  .  Condition  (3.18)  holds  by  (3.11),  and  it  follows  from  (3.41)  and  (3.19)  that 

h  c  '^1'^ 

i;inax7({||  -  Pn)\\y^  >  ^^^})  <  const  t. 

Since  £  can  be  chosen  arbitrarily  small.  (3.40)  follows. 

This  completes  the  proof  of  Theorem  3.2. 

Corollary  3.2.1:  Let  {«„  }  l)e  a  se((uenc('  of  positive  numbers  such  thcit  «„  =  o{n).  Suppose 
that  6  CLT{P)  and  |A’'„  —  n|  =  ).  Then 

1/2 

liA,  ,v,.  -  Allr  =  o„(— ). 

n 

To  prove  the  corollary,  it  is  ('noup;h  to  apply  Theorem  3.2  to  any  s('(iu('nc('  {/>„  }  such  that 
(i.„  =  ()(b„). 

Corollary  3.2.2:  Suppose-  that  T  €  CLT(P)  and  |A’’„  —  /t|  =  Op{ii).  Then 

-  A,lk  =  o,.(»-'/2). 

Corollary  3.2.3:  Unde-r  the  conditions  of  Corollary  3.2.2. 

||Ary^(A,.,v„  ~  P„)-  «‘/'(P„  -  Pn)\\T  =  o,.(l). 

Mon'ovor,  under  th('  ronditions  of  Tlu'orcixi  3.2, 


iivy"(p„.v„ 


-p„) A, 


and  xinder  the  conditions  of  Corollary  3.2.1 


-p.) 


n^l\Pn-Prr)y  =  0,{ 


Indeed, 


\\nIJ'Hp.,.n.  -  p„)  -  n'i-‘{p„  -  r„)\\r 

<  -  >.''''KI|A,.w„  -  P.Wr  +  llA.  -  P.Wr)  +  ii''"l|A,,v„  -  A,ll^. 

Under  the  conditions  of  Corollary  3.2.2, 

n 


17 


Under  the  conditions  of  Theorem  3.2  we  have 

"  =  »;)(«« n”‘ 

Theorem  3.1  implies  that  —  P„||:^}  is  stochastically  bounded,  and  the  result  follows 

from  Corollary  3.2.2  and  Theorem  3.2.  The  last  case  is  quite  similar. 

Corollary  3.2.4-  Under  conditions  of  Corollary  3.2.2, 

\/ —  Pn),  \Aj(Ai  —  Pn))  0, 

and  there  exists  a  P-Brownian  bridge  Gi>{f).,  f  G  P  defined  on  the  j^robability  space 
(fXS,7)  stich  that  G'/->  G  C\„i{T)  7  -  a.s.  and 

d~f{\/N7,(P„.N„  -  P„),Gi>)  — »  0 


in  probability  (7). 

4.  MATHEMATICAL  JUSTIFICATION:  CONVERGENCE  RATES  FOR  THE 
SEQUENTIAL  BOOTSTRAP 

In  what  follows  we  apply  the  rt'sults  of  Sc'ctioii  3  to  the  empirical  measures  based  on 
sequential  resampling  bootstrap.  We  start  with  the  following  theorem  summarizing  the 
properties  of  the  empirical  measures  in  this  case. 

Theorem  4.1.  Suppose  that  P  G  CLT{P)  and  h't  Pn,N„  be  the  empirical  measure  bcised 
on  a  secpiential  bootstrap  sampl(\  Then 

||P,„v,.  -P,lk  =  o,(/rVM 

and 

ll-'vy-in-,.,.  -Pn)-  n^'HPn  -  Pn)\\:F  = 

Indcoxl,  in  this  case  we  have,  l)y  (2.9).  |jV„  — /j|  =  and  Corollaries  3.2.1  and  3.2.3 

imply  the  result. 

In  particular,  we  have  (se<'  Corollary  3.2.4) 

d-,X-,{V^,{Pn.N,.  -  Pn),  MPn  -  Pn))  ^  0 

and 

d^{^,(Pn.N,.-Pn),Gl>)^0, 

in  probability  (7),  when'  CGd.f),  f  ^  IP  if'  a  P-Brownian  bridge,  defined  on  the  prol)ability 
space  (fi,E,7)  and  such  that  Gi>  G  Ci,u(P)  7  -  a.s.. 

For  simplicity,  we  consider  in  what  follows  only  the  case  A'  =  PU  In  this  case,  it  is  natural 
to  take  P  :=  {/(_oo,t]  :  t  G  P*},  so  that  general  empirical  measures  considered  above  turn 
out  to  be  classical  empirical  distribution  functions.  Denote 

Fit)  :=  P{{-ooj])  =  f  /(_oo.<]</P, 

Jr' 
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Fn{t)  ~  Pni(-OOj])  =  /  1(^00 j]dPn 

Jr^ 

and 

Fn,N{i)  •=  Pn,N((-OOj])  =  /  I^_^^t]dPn,N ■ 

Jr^ 

We  also  use  the  abbreviation  Fn  :=  Fn,,i. 

In  what  follows  ||  •lU  denotes  the  sup-norm  of  a  function  from  an  interval  J  C  into 
(in  particulcir,  there  could  be  .7  =  7?^ ). 

Since,  by  the  Kolmogorov- Donsker  theorem,  we  have  T  €  CLT{P)  for  all  Borel  proba¬ 
bility  measures  P  on  R^  (where  P  =  t  €  72*}),  we  have  the  following  statement. 

Theorem  4.2.  For  any  distribtition  function  F  on  72*, 

||F„,,v,. -F„|U  =  0„(n-'*/'‘) 

and 

l|A'y'''(F„,,v,.  -  F„)  -  n'/2(F„  -  F„)||oo  =  0„(n-‘/‘). 

In  the'  case  of  uniform  distribution  F  on  [0.1],  we  get  as  a  trivial  corollary  that  the 
sequence'  of  stochastic  processes 


converges  weakly  (say,  in  the  space  7'^([0,  Ij)  or  Z)[0, 1])  to  the  standard  Brownian  bridge 
process 

B{t)  :=  -wit)  -  tiu(l),  t  e  [0,1], 

to  being  the  standard  Wiener  jjrocess.  More  generally,  if  F  is  a  continuous  distribution 
function  in  72*,  then  the  limit  is  the  process  {DoF)(t.)  =  B(F{t)),  7  €  72*.  The  jse  facts  easily 
imply  justification  of  the  se'ciueiitial  n'sampling  l)ootstrap  for  a  variety  of  statistics  0„ ,  which 
can  1)0  represomte'd  as  =  T(Fn)  with  a.  compactly  (Hadamard)  differentiable  functional 
T.  More  precisely,  T  is  a  functional  (or,  more  genc’rally,  an  operator  with  values  in  a  linear 
normed  space)  G  i— >  T(G),  defined  on  a  set  Q  of  distribution  functions  G.  T  is  supposed 
to  be  compactly  differential )le  at  F  tangentially  to  the  space  of  all  uniformly  bounded  and 
uniformly  continuos  functions  (see,  e.g.,  Gill  (1989)).  For  such  statistics,  we  have 

r(F„,,vj-r(F„)  =  o,.(n-*/^), 

proving  the  first  order  asymptotic  correctness  of  the  secpicntial  resampling  approach.  These 
observations  can  b('  applied,  for  instance',  to  the  operator  G  >-*  G'~',  defined  by 

6'-' (7)  :=  inf{.r  €  72*  ;  G{x)  >  7), 

and  taking  a  distribution  function  to  its  cpiantile  function  (see,  e.g.,  Fernholz  (1983)  or  Gill 
(1989)  for  compact  differentiability  of  such  operators;  see,  also,  Koltchinskii  (1995)  for  the 
extensions  to  the  multivariate  case).  Specifically,  if  F  is  continuously  differentiable  at  a 
point  X  G  72*  with  F'{x)  >  0,  then 


(4.1) 
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where  t  =  F{x).  In  fact,  we  obtain  below  a  bound  for  quantiles,  sharper  than  (4.1)  (see 
Theorem  4.4). 

Another  interesting  application  of  Theorem  4.1  is  related  to  a  recent  work  of  Dudley 
(1992).  Namely,  we  show  that  in  Theorem  4.2  the  sup-norm  can  be  replaced  by  a  stronger 
one,  the  so  called  p-variation  norm.  For  a  function  /  from  an  interval  J  C  into  and 
for  p  >  0,  define  the  p-variation  of  /  as 

m 

■yp(/)  :=  sup|^  |/(^^)  -  ./'(J’i-i  )|'’  :  •(■()  <  <  •  •  •  <  x,n  €  J,  xq  €  J,  m  =  1,2, ...  |. 

Z  =  I 

Let 

IT,  :=IT„(.7)  :={/:/-,(/)< +oo}. 

We  introduce  the  following  norm  on  IT,  for  p  >  1  (see  Dudley  (1992)): 

ll/lli,p=ll/IU + 

Theorem  4.3.  For  any  (listril)uti()n  function  F  on  i?.^  and  for  all  p  >  2 

II -  F„||[,]  =  0,,(n~  ‘/‘^) 

and 

||iVy^(F„,;V,.  -  F„)  -  n7/’^(F„  -  F„)||[,]  = 

Proof:  Assume  that  .7  =  72‘,  take  q  <2  and  consider 

<!}• 

By  Theorem  2.1  of  Dudley  (1992).  F,,  €  CLT{P)  for  any  Borel  proljability  measure  P  on 
7?,'.  On  the  other  hand.  L.C.  Young’s  duality  inecpialities  for  p- variation  norms  (see  Dudley 
(1992),  Proposition  3.7)  imply  that  for  any  p  >  1  and  q  >  I  with  p~'  -f  q~'  =  1  we  have 

||F„,,v„  -  F„||[,]  <  const  ||F„jv,.  -  PnWr,. 

and  Theorem  4.1  implies  the  result. 

Finally,  we  improve  bound  (4.1)  for  empirical  quantiles. 

Theorem  4.4.  Suppose  that  F  is  an  absolutely  continuous  distribution  function  in  R^ . 
Let 

.4  :=  sup{.c  :  F{x)  =  ()},  D  :=  inf{.r  :  F{x)  =  1}. 

Supi)ose  the  density  of  F  is  a  stricly  positive  continuous  function  in  (.4,77).  Then  for  all 

*e(0,i) 

(4.2) 

For  any  closed  interval  J  C  (0, 1) 

sup|F,7;vJt)  -F-‘(t)|  =  Op(n-'‘/'iyi^).  (4.3) 

t£j 


L 
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If,  moreover,  A  >  — oo,  B  <  +oo  and  the  density  of  F  is  a  strictly  positive  continuous 
function  on  [A,  B],  then  (4.3)  holds  with  .7  =  [0, 1]. 

In  order  to  prove  this  theorem  we  need  several  hmimas.  The  first  one  gives  an  inequality 
for  qiiantiles.  It  requires  some  notations.  Given  li  :  [0,1]  /?*,  denote 

ijOh{s;S)  :=  sup{|//.(n)  —  //(i))|  :  |»-  —  ■‘^\  <  j'^  —  s\  <  6,  ti,v  G  [0,1]}, 

(MhiS)  :=  sup  6  >  0. 

•f€[0,ll 

Given  a  distribution  function  G',  we  set 

/)(G):=  sup  ]G(G-'(/))-<|. 


Lemma  4.1.  Suppose'  that  F  is  a  uniform  distrihutioii  function  on  [0, 1].  Tlu'ii.  for  all 
(lis  trihut  ion  functions  G  and  H  on  [0, 1]  and  for  all  t  G  [0, 1],  wo'  have 

]G-'(t)-i7-'(/)] 

<  ua-rit;  IIG  -  F]]oo  +  \\H  -  Fl]oo  +  HG)  +  HH)) 

+  \\H  -G\\oo  +  HG)  +  f>(H).  (4.4) 

Proof:  First,  note  that  for  all  t  G  [0, 1] 

[G  *(t)  — f|<|G  '(t)  — G'(G'  '(/))|4-]G(G'  ' (t))  — /]  <  ]]G  —  F]|oc  +  <^(G'). 
Similarlv, 

|77-'(/)-F-'(/)]  <  l|F-F||oo  +  <^(77). 

It  follows  that 

\G-'(t)  -  <  ]]G  -  F]]^  +  \\H  -  Flloo  +  HG)  +  d{H).  (4.5) 

On  the  other  hand, 

|G-'(0-^“'(*)l  <  \{G-F){G-'{t))-(H-F){H-'(t))\  +  \GiG-'{t))-H{H-'{t))\.  (4.6) 
Since,  l)y  (4.5), 

|(G-F)(G-'(/))-(77-F)(77-'(/))| 

<  KG'  -  F)(G'-'  it))  -  (G  -  F)(H-'{f))\  +  1(G  -  F){H-'{t))  -(H-  F)(H-'{t))\ 

<  u{t-  \\G  -  F\\^  +  \\H  -  Flloo  +  HG)  +  <!)'(F))  +  ||G  -  i7||oo 

and 


|G(G'‘(<))  -  <  \G{G-\t))  -  (1  +  |(  -  <  6(G}+t,{H). 
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(4.6)  implies  (4.4),  which  completes  the  proof. 

Next  we  need  the  following  Ijc^unds  for  the  continuity  modulus  of  the  standard  empirical 
process 

based  on  a  sample  from  the  uniform  distribution  on  [0, 1].  They  seem  to  be  known,  although 
we  have  not  found  the  exact  reference.  See  Stute  (1982)  and  Shorack  and  Wellner  (1986) 
for  some  other  results  on  oscillatory  behaviour  of  empirical  processes.  We  sketch  the  proof 
here  for  completncss. 

Lemma  4.2.  For  any  C  >  0  and  for  all  t  G  [0, 1] 

Euj^JUCn.-'^'^)  =  0(n- */'').  (4.7) 

Ea-vJC'n -'/■')  =  0(n-'/*  (4-8) 

Proof:  By  tlu'  Koinlos-Major-Tusnndy  tlu'ornn  (s('(\  (\g.  Shorack  and  W('lln('r(  198G) ), 
there  ('xists  a  s('queiic('  {Df,}n>\  of  Brownian  l)ridges  on  [0,1]  and  constants  D  >  0,  A'  > 
0,  ^  >  0  such  that 

7{A„  >  n-'/'^(.r  +  I>log70}  <  .r  >  0, 

where  :=  ||C„  —  i?n ||oo-  If  follows  that 

AT 

£'max(jt*/'^  A„  —  Dlog7/.,0)  <  -j, 

u 

and  since' 

<  Dlogti  +  niax(//V‘‘^ A„  —  Dlog/?. 0), 

we  have 

AA,,  =  0(/7-'/nogn). 

Note  that 

a;c„(/;C'77-'/2)  <  u;b,.(/:C'77-'/2)  +  2A„  . 

cind 

-\-„(C77-'A)  <  c.-„..(C'77-'A)  +  2A„. 

Therefore  it’s  enough  to  show  that 

EuiAt-X'u.-'^'^)  =  0(7?-'/^) 

and 

EuB{Cn~^^^)  =  \/logn), 


which  follow  from  the  representation  B{t)  =  ■ti>(t)  —  t.iu(l)  and  well  known  properties  of  the 
Wiener  process. 
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Proof  of  Theorem  4.4:  Without  loss  of  generality,  we  can  and  do  assume  that  F  is 
uniform  on  [0,1]  (in  the  general  case,  the  transformation  Xi  F(Xi)  should  be  used  to 
reduce  the  problem  to  the  uniform  one).  Then  we  use  Lemma  4.1  to  get  the  following  bound: 


^  ““  ^\\oo  +  F\\oo  +  )) 

+  ||^n  ““  ^n||oo  +  HFu.Nn  )  +  (4.9) 

Note?  that,  by  visual  limit  theorems  for  empirical  processes  and  their  bootstrapped  versions. 


\\F,.-FU 

<  lie,  -  e,iioc  +  lie,  -  f\\^  =  o,,('n-'/"). 

(4.10) 

By  Theorem  4.2. 

lie,.,v„  -  e.iioo  =  0, ,(«“■’/■•). 

(4.11) 

It  follows  that  w('  also  have' 

T 

II 

_8 

te 

1 

(4.12) 

Now  we  are  going  to  show  that 

i){Fn)  =  0,,{n  'logn) 

(4.13) 

and 

HFn.Nn)  =  Oj,{ir^  logn). 

(4.14) 

Indeed,  ^(Fn)  is  h'ss  than  or  ('(pial  to  the  largest  jump  of  F,,,  which  is  ('qual  tc 


niaxi<j<„  Uj 


in) 


n 


where 


We  have 


k^i 


Eexpl.J'")  =  n  E-'>:i>U(,v„,=,v,,  I  =  (1  +  ^)"  < 

fc=l 

ThenTore,  for  C  >  1 

j{  max  >  CTog77} 

Vl<)<n  J  ~  ’ 


<  exp{— CTogn}  ??.  E exp{ui"^]  <  e®  =  o(l). 


7-a.e.  and  (4.13)  follows. 
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The  proof  of  (4.14)  is  quite  similar.  In  this  case  we  set 

^5  *  Xi  =  }  ’  i  =  •  •  • . 


and  get 


We  have 


where 


maxi<_,<„  Uj 


(4.15) 


7 {  max  >  Clog?/}  <  7{  umx  >  CTognj  +7{.V„  >  2??.},  (4.16) 


we  easily  obtain 


C exp 


7 {  max  ?>]"’  >  C  log  ??. }  =  o(  1 ), 


and  (4.14)  follows  from  (4.15)-(4.17)  and  (2.9). 

By  (4.9)-(4.14),  in  order  to  prove  (4.2)  it  remains  to  .show  that  for  all  t  e  (0.1) 

;md  in  ord('r  to  prove  (4.3)  wv  hav('  to  show  that 

^7.'„_/,-(<^'^'“'/'^)  =  Op(/;“  */'^  \/logn). 

We  show  here  only  (4.19),  since  the  proof  of  (4.18)  is  (piite  similar.  Note  that 


(4.18) 


(4.19) 


where 


;=  :  |(  -  <  2Cn-'/'‘,  s,(  €  10,1]}. 


It  follows  from  Lcuunni  4.2  that 


n 


Now  we  use  Lemma  3.5  with  m  =  ri  and  with  function 

i/’(;c)  :=  const  y/\og{x  +  e'^) 
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(which  satisfies  all  conditions  required  in  Lemma  3.5),  to  get 

n 

nEE  \\Pn  -  Pn\\:F„  =  II  -  Pn)\\j^n  <  const  iin), 

I 

which  implies 


EElo.  (C'n-‘/2)  =  EE  II a.  -  PnWr,,  =  0(n-^/'‘x/i^). 

Combining  this  bound  with 

EEu:r„-F{Cn-^^^)  = 

(which  follows  from  (4.8)),  we  get 

EE  <  EE  u.’,,^_/,_(C7/-'/A  +  EE  ujr„-F(Cii-'^'^)  =  0(77-'‘/'^  V^k^). 

This  implies  (4.19),  comph'ting  tin*  procif. 

It  worth  noting  that  the  rate  77 “■*/'’  (up  to  a  logarithmic  factor)  in  Theorem  4.4  is  exactly 
the  same  as  in  the  remarkai)le  Baluulur-Kiefer  representation  for  empirical  cpiantiles  (see, 
e.g.,  Shorack  and  Wellner  (19SG)). 

5.  CONCLUDING  REMARKS 

Having  seen  in  the  preceding  investigation  that  the  “distance”  l)etween  the  sequential 
bootstrap  and  the  common  bootstrap  of  size  n  is  at  most  of  order  (up  to  a  logarith¬ 

mic  factor),  it  is  now  natural  to  ask  about  the  second-order  correctness  of  the  sequential 
bootstrap  as  compare<l  to  the  common  bootstrap.  Our  preliminary  investigation  along  the 
lines  of  the  Hall-Mamnien  pfiper  (1994)  seems  to  indicate  that  the  second-order  correctness 
of  the  sequential  bootstrap  does  indeed  go  through.  Further  work  in  this  direction  is  in 
progress  and  will  be  published  later. 
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